Purpose: How the attention mechanism helps AI focus on the most relevant parts of the input.

Exercise (1)

---

• Identify the most important words in a sentence that describe an image and learn how the attention mechanism helps AI focus on the most relevant information.

Exercise (2)

---

• Watch a video to understand how the attention mechanism works. Explore how AI uses important words to generate better responses.
• Links: https://www.youtube.com/watch?v=2RdK6k45koY , https://www.youtube.com/watch?v=gGK7b7j4BiQ

Solutions

---

• Link to view the solution for exercise-1
• Link to view the solution for exercise-2

1. Explain that human attention naturally skips background details to focus on key elements, and AI uses the attention mechanism to do the same.
2. Guide students through Exercise 1 by asking them to highlight key words in a descriptive sentence before revealing how an AI calculates attention weights for each word.
3. Discuss how attention allows AI models to process entire long sentences at once rather than reading rigidly word-by-word.
4. Points to Ponder:
   • When looking at a photo of a dog in a park, which pixels should receive high attention from an AI trying to identify the dog?
   • How is an AI's mathematical attention different from human attention when listening to a teacher in a noisy classroom?